This invention relates to a sensor system for sensing radiation from a scene. It also relates to a method of sensing such radiation.
Radiation sensor systems are well known in the prior art. They find widespread application, for example, in portable consumer video and digital cameras and also in thermal imagers as employed by emergency services.
A typical system incorporates a sensor comprising a two-dimensional array of elements, each with an associated signal processing circuit. Radiation from a scene is projected onto the array where each element responds via its associated processing circuit with an output Sk as given in equation [1]; the index k here is used to identify elements uniquely, i.e. Sk is the output from the kth element circuit. The output Sk includes unwanted artefacts which arise either from the scene itself or are generated within its associated element or processing circuit:
Sk=Ak(Rfk,Rpedk)+Bk+Nkxe2x80x83xe2x80x83Eq. 1
where
Sk=output generated from kth element via its associated processing circuit;
Ak=kth element responsivity function;
Rfk feature information or scene contrast radiation from the scene received at the kth element;
Rpedk=background radiation from the scene received at the kth element;
Bk=offset signal generated within the kth element and its associated processing circuit; and
Nk=noise signal generated within the kth element and its associated processing circuit.
The outputs Sk from each element are combined to provide a sensor signal. The sensor including its associated circuits may be based on charge coupled devices (CCDS) or metal oxide semiconductor (MOS) devices. When MOS devices are employed in particular, it is found that there is an undesirable variation in element responsivity function Ak, namely the elements have differing responsivities and give different outputs Sk in response to the same received radiation intensity. This variation is often larger than that of sensors incorporating charge-coupled devices (CCD). It has prevented widespread use of sensors incorporating MOS devices in consumer video cameras in preference to sensors incorporating CCDs despite a long-felt want to do so in order to benefit from the compatibility of MOS detection and processing circuitry. In this connection, providing power supply and control signals for operating MOS devices tends to be less complex and less expensive compared to providing them for operating corresponding CCDs; this is because circuit parameters such as supply voltages are compatible in the former case. The variation gives rise to fixed pattern noise (FPN) in the outputs Sk which results in the corresponding sensor signal depicting a speckled scene. Moreover, the elements also have differing values of the offset signal Bk amongst the elements. For sensors employed to respond to low level radiation intensity, noise Nk generated within their detector elements and associated processing circuits often becomes a problem, in particular flicker noise contributing to Nk which has a noise spectral density which increases inversely relative to frequency.
When the sensor receives radiation from the scene, the output Sk contains an unwanted pedestal component, corresponding to a general background radiation from the scene, together with a feature or contrast component which corresponds to features within the scene. This is particularly pertinent when:
(i) the sensor is detecting infra-red radiation;
(ii) the scene is at an ambient temperature of approximately 300 K; and
(iii) the temperature variations within the scene Rfk giving rise to the feature component are less than 1 K.
The pedestal component may be a factor of one thousand or more larger than the feature component. This results in poor signal contrast which may render the temperature variations difficult to identify in the outputs Sk unless further signal processing is applied thereto.
The presence of the pedestal component imposes constraints and limitations on design and performance of a sensor system for sensing emissions from a scene, especially infra-red emissions therefrom. It often results in a sensor signal representing an image of the scene which is dominated by offset errors and artefacts with respect to scene contrast.
A solution which addresses the problem of pedestal component described above is disclosed in U.S. Pat. No. 5,155,348. This describes a read-out circuit for a sensor comprising a two-dimensional array of 128xc3x97128 photodetector elements responsive to infra-red radiation where each element is connected to a respective read-out circuit. The circuit has calibration and measurement phases.
During the calibration phase, a calibration image is projected onto the elements. The image may correspond to a featureless calibration object of similar temperature to a scene to be viewed or a totally blurred featureless uniform image of the scene. Each element generates a signal in response to the calibration image and its respective circuit is arranged to store a calibration signal on a storage capacitor Cc incorporated within it corresponding to a signal generated by its respective element in response to the calibration image. This provides a correction for pedestal component across the array.
During the measurement phase, a focused image of the scene is projected onto the array and generates a measurement signal at each element. The calibration signal is subtracted from the measurement signal for each element to provide a difference signal. The difference signal is integrated to provide an output signal. The circuits produce respective output signals scanned by a multiplexer to give a compound sensor signal.
This solution reduces the dynamic range of the compound sensor signal by removing pedestal component at each element. It eases dynamic range performance requirements of remote circuits receiving the sensor signal from the multiplexer, for example allowing use of analogue-to-digital converters of 8-bit resolution instead of 12-bit resolution.
A technique for reducing FPN is described in a U.S. Pat. No. 5,373,151. The specification is concerned with an optical system which projects focused and periodically defocused images of a scene onto a multielement focal plane array for generating at each element corresponding focused and defocused signals respectively. A difference signal for each element is derived by subtracting its associated defocused signal from its focused signal. Difference signals from the elements are combined together to provide a system output in which FPN artefacts have been reduced.
Despite use of FPN correction in the prior art, it is found in practice that image quality in the prior art does not remain as good as the initial correction would suggest and it is necessary to recalibrate repeatedly. Reasons for this inconsistency are not presently disclosed in the prior art.
It is an object of the invention to provide a sensor system with more effective FPN correction.
According to the present invention, a sensor system is provided for generating a sensor signal corresponding to a filtered image of a scene, the system incorporating:
(i) detecting means incorporating a plurality of detector elements for generating first and second element signals during first and second detection phases respectively; and
(ii) processing means for deriving a difference signal from the element signals for use in generating the sensor signal,
characterised in that the processing means incorporates sensing means for sensing at least one environmental factor influencing responsivity of the elements and the system is arranged to repeat both of the first and second phases in response to environmental change.
The invention provides the advantage that the sensor system undergoes recalibration to reduce FPN when only when necessary when changes in any one of environmental factors influencing responsivity of the system occurs.
A first problem with the detection circuit described in the U.S. Pat. No. 5,155,348 and with the system described in U.S. Pat. No. 5,373,151 is that the responsivity function Ak in Eq. 1 of their detector elements is influenced by temperature. In consequence, FPN correction described in these two specifications becomes inaccurate when their detector element temperature changes. Moreover, a second problem is that variation in the function Ak of the detector elements will vary with received radiation level; for example, elements in a multielement detector array often exhibit mutually identical responsivities at relatively higher received radiation levels and mutually different responsivities at relatively lower received radiation levels. This means that FPN correction determined at relatively higher received radiation levels will be inaccurate if used at lower received radiation levels. This temperature and radiation level sensitivity of the responsivity function Ak of elements is not appreciated in the prior art.
A conventional approach to addressing the first problem is to stabilise multielement detector array temperature, for example by using temperature controlled elements thermally coupled to the detector array. The temperature controlled elements may, for example, comprise Peltier elements although these have a disadvantage of associated bulk, cost, high power consumption and slow thermal response. Moreover, many conventional optical systems incorporate iris components for stabilising global detector array illumination; such iris components are frequently employed in conventional 35 mm film cameras incorporating automatic exposure control. These iris components are often precision optical parts which are prone to mechanical wear.
Although FPN reduction is described in the U.S. Pat. Nos. 5,155,348 and 5,373,151, frequent regular recalibration to reduce FPN is undesirable because:
(i) interruption of sensor signal occurs during an associated calibration phase; and
(ii) mechanically actuated optical components for providing blurred and unblurred images are prone to mechanical wear and related unreliability problems.
When the sensor is outputting at high frame rates, for example 25 images/second in video applications, periodic interruption of its sensor signal for recalibration is undesirable where the calibration phase is substantially equal or longer in duration than the measurement phase. Mechanical actuation of optical components is a relatively slow process, for example actuating a 50 g lens through a distance of 1 mm using a compact linear actuator to project focused and defocused images will often take 50 msec. Interruption of the sensor signal can result in flicker therein, and can, for example, result in control instability problems when the sensor signal is employed in a visual feedback control system. Visual feedback control systems comprise, for example, robotic fruit picking equipment for use in agriculture and security inspection systems.
The problems described above are reduced, according to the present invention, by arranging for a sensor adapted to execute calibration and measurement phases for reducing FPN as described in the U.S. Pat. Nos. 5,155,348 and 5,373,151 to perform its calibration phase in response to one or more environmental parameters affecting sensor characteristics changing by more than a threshold amount compared to its value when the calibration phase was most recently executed.
The environmental factors include, for example, any one of:
(i) system temperature; and
(ii) total amount of radiation received at the system from the scene.
Thus,
(i) the sensing means may be responsive to at least one of temperature and illumination at the detecting means; and
(ii) the system may be arranged to repeat both the first and second phases in response to change in at least one of temperature and illumination by more than a preset value.
FPN reduction in the sensor is achievable by making one of the images a local spatial average image of the scene. The local spatial average image is obtainable by making one of the images a diffuse image of the scene. The system may, therefore, incorporate projecting means for projecting scene images onto the detecting means so that at least one element signal is generated during a diffuse image phase.
The projecting means may incorporate actuating means for interposing radiation scattering means between the scene and the detecting means. This provides the advantage of a practical mechanical arrangement for generating diffuse images. For example, the scattering means may be interposed for generating diffuse images with relatively low mechanical precision compared to the accuracy with which the projecting means must be maintained relative to the detecting means for projected a finely focused image thereonto; thus, the system can be converted from one projecting a finely focused image to one projecting a blurred image merely by imprecise interposition of the scattering means without disturbing position of the projecting means relative to the detecting means.
The actuating means may comprise at least one of solenoid means and means for translating and rotating the scattering means to interpose it between a scene and the detecting means. This provides the advantage that the actuating means can be adapted to suit various mechanical constraints imposed by external packaging of the system, for example its shape of enclosure, cabinet or casing.
The scattering means may incorporate at least one of the following: a ground glass plate, a translucent plastic sheet, a sheet of tracing paper, a microprism sheet, a Fresnel plate and a phase plate. This provides the advantage of a selection of practical integers for use in generating diffuse images.
The projecting means may be arranged so that radiation corresponding to that receivable on one element from a focused image is received by between two elements and 64% of the elements when the image is diffuse. This provides the advantage of being a useful practical range of blurring for achieving FPN reduction.
The system may be adapted so that:
(i) it incorporates projecting means for projecting scene images onto the detecting means;
(ii) the projecting means is arranged to project a local spatial average image onto the detecting means during one of the phases and a focused measurement image during another of the phases; and
(iii) the processing means is arranged to generate the sensor signal from the element signals derived from the local average and measurement images.
This provides the advantage that FPN reduction is achievable by incorporating projecting means into the system for projecting a local spatial average image of the scene onto the detecting means.
The measurement image may be projected onto the detecting means prior to the local spatial image, thereby providing image tone reversal in the sensor signal. This provides the advantage of the system simultaneously performing FPN and tone reversal simultaneously. Tone reversal here is defined as regions of the scene which are more radiation emissive than other regions thereof being represented as relatively darker areas in the sensor signal compared to the other regions represented in the signal.
The projecting means may incorporate a liquid crystal spatial light modulator configured to be controllable between a first substantially non-scattering state and a second diffusing state as regards transmission of radiation from the scene to the detecting means in both cases. This provides the advantage of being a simple approach for generating diffuse images without needing to use mechanically actuated components which are prone to wear.
The liquid crystal modulator may be a polymer dispersed liquid crystal device (PDLC) having scattering and non-scattering states which are selectable in response to a control potential applied to the device. This provides the advantage of being a simple device which is conveniently switchable from one state to another and which does not require polarising filters to operate unlike other types of liquid crystal devices. Moreover, it responds more rapidly and is more compact than mechanically actuated diffusing shutters.
The system may be incorporated into one or more of a digital stills camera, a video camera, a personal electronic organiser and a mobile telephone. When the system is employed in a digital stills camera or a video camera, it provides FPN correction which allows CMOS detectors to be employed in these cameras. Personal electronic organisers often operate with limited supply potentials available for electronic components incorporated therein from their batteries; the invention enables a sensor system employing CMOS devices to be incorporated into the organisers, for example for imaging and scanning documents presented to the organisers. The invention incorporated into a mobile telephone provides data entry of images into the mobile telephone thereby providing a videophone; the invention provides the advantage of enhanced image quality, especially when the mobile telephone is used infrequently and is subject to extreme temperature variations when in use.
In another aspect, the invention provides a method of providing from a sensor system a sensor signal corresponding to a filtered image of a scene, the method including the steps of:
(a) monitoring a multielement detector incorporated into the system to measure environmental factors influencing its responsivity;
(b) arranging for a projector in the system to project a first image onto the detector during a first phase to provide a first signal at each element;
(c) recording the first signal of each element;
(d) arranging for the projector to project a second image onto the detector during a second phase to provide a second signal at each element, at least one of the first and second images being a scene image;
(e) deriving a respective difference signal from the first and second signals of each element from the second element signal of each element to provide a difference signal;
(f) outputting the difference signal for each element collectively to provide the sensor signal;
(g) monitoring the detector environment to detect changes influencing its responsivity; and
(h) repeating either steps (d) to (g) or steps (b) to (g) according to whether environmental change is or is not detected to be less than a prearranged threshold.
In a further aspect, the invention addresses a problem of generating a local spatial average image for use in performing FPN correction. In the prior art, for example in the U.S. Pat. No. 5,155,348 and in the system described in the U.S. Pat. No. 5,373,151, a local spatial average image is created by projecting a defocused image of a scene. This involves actuating optical projection components between states where they project focused and defocused images.
In the further aspect, the invention provides a sensor system for generating a sensor signal corresponding to a filtered image of a scene, the system incorporating:
(i) detecting means incorporating a plurality of detector elements for generating first and second element signals during first and second detection phases respectively; and
(ii) processing means for deriving a difference signal from the element signals for use in generating the sensor signal,
characterised in that the system incorporates projecting means for projecting scene images onto the detecting means (528) so that at least one element signal is generated during a diffuse image phase.
The invention provides the advantage of being a practical approach to generating a local spatial average image. For example, a diffuse image may be generated during a diffuse image phase by interposing scattering means with relatively low mechanical precision compared to the accuracy with which projecting means must be maintained relative to the detecting means for projected a finely focused image thereonto; thus, the system can be converted from one projecting a finely focused image to one projecting a blurred image merely by imprecise interposition of the scattering means without disturbing position of the projecting means relative to the detecting means.